Papers by Leonardo F. R. Ribeiro
Benchmarking Deflection and Hallucination in Large Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks overlook conflicts between visual and textual evidence and the importance of generating deflections when incomplete knowledge is retrieved. |
| Approach: | They propose a dynamic curation pipeline that preserves benchmark difficulty over time . they propose 'vlm-DeflectionBench' benchmark to probe model behaviour under conflicting evidence . |
| Outcome: | The proposed benchmarks overlook conflicts between visual and textual evidence and are prone to obsolescence . the proposed benchmark is based on 2,775 samples spanning diverse retrieval settings . |
Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning (2025.findings-naacl)
Copied to clipboard
Hyundong Justin Cho, Karishma Sharma, Nicolaas Paul Jedema, Leonardo F. R. Ribeiro, Jonathan May, Alessandro Moschitti
| Challenge: | Language models are biased towards generic outputs as they are trained to align to an aggregate preference to be generally useful. |
| Approach: | They propose a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user. |
| Outcome: | The proposed method achieves favorable win rates on pairwise comparisons with the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles. |
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments demonstrate that integrating relevance feedback directly into neural re-ranking models improves their performance. |
| Approach: | They propose a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. |
| Outcome: | The proposed approach outperforms all other methods by 5.2% on large-scale information retrieval datasets. |
Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent graph-to-text models generate text from graph data using global or local aggregation . global node encoding allows explicit communication between two distant nodes, but fails to capture long-range relationships. |
| Approach: | They propose to combine global and local aggregation to learn node representations . they propose to use global and locally encoding to learn contextualized node embeddings based on graph data . |
| Outcome: | The proposed models outperform state-of-the-art models on two graph-to-text datasets by 18.01 and 63.69 points. |
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA (2022.aacl-demo)
Copied to clipboard
Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner, Sewin Tariverdian, Hao Zhang, Kexin Wang, Hossain Shaikh Saadi, Leonardo F. R. Ribeiro, Iryna Gurevych
| Challenge: | Question Answering (QA) systems rely on deep neural networks, which are difficult to interpret by humans. |
| Approach: | They propose an interpretable model that provides an explanation infrastructure for comparing models based on saliency maps and graph-based explanations. |
| Outcome: | The proposed methods can be used to compare models based on saliency maps and graph-based explanations. |
NeoQA: Evidence-based Question Answering with Generated News Events (2025.findings-acl)
Copied to clipboard
| Challenge: | Evaluating Retrieval-Augmented Generation (RAG) in large language models is challenging because benchmarks can quickly become stale. |
| Approach: | They propose a benchmark to evaluate Retrieval-Augmented Generation (RAG) in large language models (LLMs) using timelines and knowledge bases of fictional news events and entities to prevent LLMs from leveraging pretraining knowledge. |
| Outcome: | The proposed benchmark prevents LLMs from leveraging pretraining knowledge and ensures that no prior evidence exists in their training data. |
FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that abstractive summarization approaches generate summaries that are not factually consistent with the source document. |
| Approach: | They propose a method that decomposes the document and summary into structured meaning representations (MRs) MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form . |
| Outcome: | The proposed method outperforms existing methods on benchmarks for factuality evaluation. |
RAGferee: Building Contextual Reward Models for Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
Andrei Catalin Coman, Ionut Teodor Sorodoc, Leonardo F. R. Ribeiro, Bill Byrne, James Henderson, Adrià de Gispert
| Challenge: | Existing Reward Models (RMs) struggle in Retrieval Augmented Generation settings. |
| Approach: | They propose a method that repurposes question-answering datasets into preference pairs that prioritise groundedness over stylistic features. |
| Outcome: | The proposed method surpasses existing RMs trained on larger general corpora with an absolute improvement of +15.5%. |
RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models fail to selectively refuse to answer based on flawed context, study finds . current benchmarks fail to evaluate complex capabilities like selective refusal . |
| Approach: | They propose a framework that generates diagnostic test cases through controlled linguistic perturbation. |
| Outcome: | The proposed framework employs 176 perturbation strategies across six categories of uncertainty and three intensity levels. |
Enhancing AMR-to-Text Generation with Dual Graph Representations (D19-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a linguistically-grounded semantic formalism that represents the meaning of a sentence as a directed graph. |
| Approach: | They propose a graph-to-sequence model that encodes different but complementary perspectives of the structural information contained in the graph. |
| Outcome: | The proposed model achieves state-of-the-art results on two AMR datasets. |
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | Recent advances on abstractive summarization have led to fluent summaries, but factual errors in generated summary still severely limit their use in practice. |
| Approach: | They evaluate summaries produced by state-of-the-art models via crowdsourcing and show that factual errors occur frequently. |
| Outcome: | The proposed models can detect errors and reduce them by reranking alternative summaries. |
Structural Adapters in Pretrained Language Models for AMR-to-Text Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have advanced graph-to-text generation, but efficient encoding of graph structure is challenging because of the nature of the data. |
| Approach: | They propose a method to encode graph structure into pretrained language models by training only graph structure-aware adapter parameters. |
| Outcome: | The proposed method outperforms the state-of-the-art on two AMR-to-text datasets, training only 5.1% of the adapter parameters. |
GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown consistent improvements across many tasks requiring natural language understanding, coding, mathematical or logical reasoning . |
| Approach: | They propose to use GaRAGe to evaluate whether LLMs can identify relevant grounding when generating RAG answers. |
| Outcome: | The proposed model over-summarises rather than ground answers strictly on annotated relevant passages, or deflects when no relevant grounding is available. |
DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality (2026.acl-long)
Copied to clipboard
Yukun Huang, Leonardo F. R. Ribeiro, Momchil Hardalov, Bhuwan Dhingra, Markus Dreyer, Venkatesh Saligrama
| Challenge: | Existing fact-checkers usually target general-domain atomic claims . citation-grounded fact- checking ignores claims without explicit citations . |
| Approach: | They propose to use a benchmark to test whether claim-level factuality is transferable . they instantiate **Audit-then-Score** as a versioned DRR factualism benchmark . |
| Outcome: | The proposed benchmark outperforms the best prior deep-research and traditional fact-checkers by 14.3 and 24.9 points. |
A Neural Graph-based Local Coherence Model (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Entity grids and entity graphs are two frameworks for modeling local coherence . many approaches to local cohesion modeling rely on entity relations between sentences . |
| Approach: | They propose to use Relational Graph Convolutional Networks to encode entity graphs for measuring local coherence. |
| Outcome: | The proposed model outperforms the neural grid-based model on two coherence evaluation tasks while using 50% fewer parameters. |
XRAG: Cross-lingual Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | XRAG evaluates the generation abilities of LLMs in cross-lingual RAG settings where the user language does not match retrieval results. |
| Approach: | They propose a benchmark to evaluate the generation abilities of LLMs in cross-lingual RAG settings where the user language does not match retrieval results. |
| Outcome: | XRAG is a benchmark designed to evaluate the generation abilities of LLMs in cross-lingual RAG settings where the user language does not match retrieval results. |
UKP-SQUARE: An Online Platform for Question Answering Research (2022.acl-demo)
Copied to clipboard
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, Iryna Gurevych
| Challenge: | Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats and require different model architectures and setups. |
| Approach: | They propose an extensible online QA platform that allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
| Outcome: | The proposed tool allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translation (2026.acl-long)
Copied to clipboard
Shaomu Tan, Dawei Zhu, Ke Tran, Michael Denkowski, Sony Trenous, Leonardo F. R. Ribeiro, Bill Byrne, Felix Hieber
| Challenge: | Large language models (LLMs) have made document-level machine translation increasingly practical, enabled by long-context modeling and strong generation quality. |
| Approach: | They propose to use document-level MT followed by segment-level refinement to find the strongest and most stable improvements across six LLMs and seven language pairs. |
| Outcome: | The proposed method outperforms error-specific prompting and evaluate-then-refine schemes in document-level translation. |
Generating Summaries with Controllable Readability Levels (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current text generation approaches focus on a specific readability level, resulting in texts that are not customized to readers’ proficiency levels. |
| Approach: | They propose to generate summaries with fine-grained control over their readability by using instruction-based readability control, reinforcement learning and lookahead to estimate readability of upcoming decoding steps. |
| Outcome: | The generated summaries with different readability levels were compared with previous methods that focus on a specific readability level (e.g., lay summarization) and a lookahead approach significantly improved readability control on news summarizing. |
Smelting Gold and Silver for Improved Multilingual AMR-to-Text Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on multilingual AMR-to-text generation has focused on data augmentation strategies that utilize generated silver AMRs, but this assumes a high quality of generated AMR. |
| Approach: | They propose to combine gold AMR with silver AMRs to generate multilingual AMR annotations. |
| Outcome: | The proposed models outperform the current state of the art for German, Italian, Spanish, and Chinese by a large margin. |